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Michael Harder, Jeff Mellen, Michael D. Ernst 

May 2003 Proceedings of the 25th international conference on Software engineering 



Full text available: 



MB; ' 



Additional Information: full citation, abstract. referar-.eas. citings 



Publisher Site 



This paper presents the operational difference technique for generating, augmenting, and 
minimizing test suites. The technique is analogous to structural code coverage techniques, 
but it operates in the semantic domain of program properties rather than the syntactic 
domain of program text.The operational difference technique automatically selects test 
cases; it assumes only the existence of a source of test cases. The technique dynamically 
generates operational abstractions (which describe obser ... 

2 Techrica^ 

effectiveness of regression testing 

Gregg Rothermel, Sebastian Elbaum, Alexey Malishevsky, Praveen Kallakuri, Brian Davia 
May 2002 Proceedings of the 24th international conference on Software engineering 

Full text available: pdf(J. : 37.MB). Additional Information: Ml .citation, abstract, references, index .teni£ 

Regression testing is an expensive testing process used to validate software following 
modifications. The cost-effectiveness of regression testing techniques varies with 
characteristics of test suites. One such characteristic, test suite granularity, involves the way 
in which test inputs are grouped into test cases within a test suite. Various cost-benefits 
tradeoffs have been attributed to choices of test suite granularity, but almost no research 
has formally examined these tradeoffs. To addres ... 
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Aj^MhodoJo^ 

M. Jean Harrold, Rajiv Gupta, Mary Lou Soffa 

July 1993 ACM Transactions on Software Engineering and Methodology (TOSEM), 

Volume 2 Issue 3 

Full text available: m od«1.12 MB) Additional Information: MsMiSQ, sbsjract, mteSSS. citings, iodex 

* terms, .review 

This paper presents a technique to select a representative set of test cases from a test suite 
that provides the same coverage as the entire test suite. This selection is performed by 
identifying, and then eliminating, the redundant and obsolete test cases in the test suite. 
The representative set replaces the original test suite and thus, potentially produces a 
smaller test suite. The representative set can also be used to identify those test cases that 
should be rerun to test the program ... 

Keywords: hitting set, regression testing, software engineering, software maintenance, test 
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4 8i-Cnterla.M6dels.fo 

May 2004 Proceedings of the 26th International Conference on Software Engineering 

Full text available: ^sSKi8S75 KB- 

j=jT Additional Information: Mi citation, abstract 

Using bi-criteria decision making analysis, a new modelfor test suite minimization has been 
developed that pursuestwo objectives: minimizing a test suite with regard to a 
particularlevel of coverage while simultaneously maximizingerror detection rates. This new 
representation makes it possibleto achieve significant reductions in test suite size 
withoutexperiencing a decrease in error detection rates. Usingthe all-uses interprocedural 
data flow testing criterion, twobinary integer linear programm ... 

Boris Vaysburg, Luay H. Tahat, Bogdan Korei 
July 2002 ACM SIGSOFT Software Engineering Notes , Proceedings of the 2002 ACM 
SIGSOFT international symposium on Software testing and analysis, volume 

27 Issue 4 

Full text available: ^pgf(322,42.KB • Additional Information: Mi .citation, abstract, references, Index.ter.rns 

Requirement-based automated test case generation is a model-based technique for 
generating test suites related to individual requirements. The technique supports test 
generation from EFSM (Extended Finite State Machine) system models. Several 
requirement-based selective test generation techniques were proposed. These techniques 
may significantly reduce a number of test cases with respect to a requirement under test, as 
opposed to a complete system testing. However, the number of test cases may st ... 

Keywords: EFSM dependence analysis, EFSM system model, interaction pattern, model- 
based testing, system testing, test suite reduction 

6 Iech.QLcal..papersL Q 
Myra B. Cohen, Peter B. Gibbons, Warwick B. Mugridge, Charles J. Colbourn 
May 2003 Proceedings of the 25th international conference on Software engineering 

Full text available: ilipdff 975.28 KB; 

M Additional Information: Ml citation, s&stract, references 

Software system faults are often caused by unexpected interactions among components. Yet 
the size of a test suite required to test all possible combinations of interactions can be 
prohibitive in even a moderately sized project. Instead, we may use pairwise or t-way 
testing to provide a guarantee that all pairs or t-way combinations of components are tested 
together. This concept draws on methods used in statistical testing for manufacturing and 
has been extended to software system testing. A cov ... 
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Paeers:.TSMLP;..Iest 

Sabine Lehmann, Stephan Oepen, Sylvie Regnier-Prost, Klaus Netter, Veronika Lux, Judith 
Klein, Kirsten Falkedal, Frederik Fouvry, Dominique Estival, Eva Dauphin, Herv6 Compagnion, 
Judith Baur, Lorna Balkan, Doug Arnold 

August 1996 Proceedings of the 16th conference on Computational linguistics - Volume 
2 

Full text available: ^.gdfi6nJ.9..KB). Additional Information: fujl.oilatjon, abstract, references, citings 

The growing language technology industry needs measurement tools to allow researchers, 
engineers, managers, and customers to track development, evaluate and assure quality, 
and assess suitability for a variety of applications.The TSNLP (Test Suites for Natural 
Language Processing) project has investigated various aspects of the construction, 
maintenance and application of systematic test suites as diagnostic and evaluation tools for 
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8 Muitjobje^ Q 
David A. van Veidhuizen, Gary B. Lamont 

February 1999 Proceedings of the 1999 ACM symposium on Applied computing 

Full text available: ^ sfff(853.'/2 KB) Additional Information: fall cation , .'-aferences. citings , index tems 



9 UsingJeM.syM 

Margaret King, Kirsten Falkedal 

August 1990 Proceedings of the 13th conference on Computational linguistics - Volume 
2 

Full text available: "fU Ddf(536,47. KB). Additional Information: full, citation, references, citings 



10 Middleware: Designing a test suite for empirically-based middleware performance 
pr&djction 

Yan Liu, Ian Gorton, Anna Liu, Ning Jiang, Shiping Chen 

February 2002 Proceedings of the Fortieth International Confernece on Tools Pacific: 
Objects for internet, mobile and embedded applications - Volume 10 

Additional Information: MLdMon, abstract, references, citings, index 



Full text available: ■ pstfft 1.03 MB) 

One of the major problems in building large-scale enterprise systems is anticipating the 
performance of the eventual solution before it has been built. This problem is especially 
germane to modern Internet-based e-business applications, where failure to provide high 
performance and scalability can lead to application and business failure. The fundamental 
software engineering problem is compounded by many factors, including application 
diversity, architectural trade-offs and options, COTS compone ... 

Keywords: COTS, component-based system, empirical results, middleware, performance 
modelling, prototype 



1 1 Vector].^ O 
D. Callahan, J. Dongarra, D. Levine 

November 1988 Proceedings of the 1988 ACM/IEEE conference on Supercomputing 

Additional Information: fail c&ation . sfrstrart:. references , citings , index 



Full text available: llSpdff 824.46 KB- 

^ terms 

This report describes a collection of 100 Fortran loops used to test the effectiveness of an 
automatic vectorizing compiler. We present the results of compiling these loops using 
commercially available, vectorizing Fortran compilers on a variety of supercomputers, mini- 
supercomputers, and mainframes. 

1 2 The camp..smajjtajk Q 
Ralph Johnson, Donald MacQueen 

January 2000 Addendum to the 2000 proceedings of the conference on Object-oriented 
programming, systems, languages, and applications (Addendum) 

Full text available: ^.pdgi9.4.1..KB} Additional Information: faiieitation, abstract., jp-dexiemis 

Although there has been an ANSI standard for Smalltalk for some years, there has not been 
a verification suite for it. This has allowed the various dialects to deviate from the standard 
without much notice. One of the first Camp Smalltalk projects was a test suite for the ANSI 
standard protocols. It has several thousand tests based on SUnit, a way of testing protocol 
conformance, and way of reusing protocol-based tests for each of the concrete classes that 
is supposed to support a protocol. 

13 Tesi.scenario.and 



http://portal.acm.org/results.cfm?coll=ACM&dl=ACM&CFro=29406108&CT 10/14/04 



Results (page 1): "skeleton description" "test suite" Page 4 of 5 

for object-oriented program testing Q 
Chun-Yu Chen, Richard Chapman, Kai H. Chang 

April 1999 Proceedings of the 37th annual Southeast regional conference (CD-ROM) 

Full text available: SpafQ 85 .,97„KB] Additional Information: Ml citation, Index terms 



1 4 A reduced test, su ite f or .pmtQCoLcon^ 
Philip J. Bernhard 

July 1994 ACM Transactions on Software Engineering and Methodology (TOSEM), 

Volume 3 Issue 3 

Full text available: ^.pd£i^4..MBS. Additional Information: feijj.cltatip.fL references, citiQ&s, index 



Keywords: heuristics 



1 5 Generating. test suites. for .software .lo^.testing 
Alberto Avritzer, Elaine J. Weyuker 

August 1994 Proceedings of the 1994 ACM SIGSOFT international symposium on 
Software testing and analysis 

Full text available: ^.:xjg1...24^Bj Additional Information: felicitation, references, cijiags, ind^.terais, review 



16 Towards a universal test suite for combinatorial auction algorithms 
Kevin Leyton-Brown, Mark Pearson, Yoav Shoham 

October 2000 Proceedings of the 2nd ACM conference on Electronic commerce 

Full text available: ^.pdf(494.90. KB). Additional Information: full. citation., references, citings, index ter.0)s 



17 ApMsMes^ 
Ping Tak Peter Tang 

September 1991 ACM SIGAda Ada Letters, Volume xi issue 7 

Full text available: * Wl ErfftUI MB) Additional Information: lull citation , abstract; indox terms 



A comprehensive set of elementary functions has been implemented portably in Ada. The 
high accuracy of the implementation has been confirmed by rigorous analysis. Moreover, we 
present new test methods that are efficient and offer a high resolution of 0.005 unit in the 
last place. These test methods have been implemented portably here and confirm the 
accuracy of our implemented functions. Reports on the accuracy of other function libraries 
obtained by our test programs are also presented. 

18 PIWG.ieMiesu]ts □ 
Dale Gaumer 

January 1990 ACM SIGAda Ada Letters , Proceedings of the working group on Ada 

performance issues 1990, volume x issue 3 
Full text available: ^pdl(6 ; 6S..fv1B). Additional Information: ML&Mion, .Index jerrns 



19 An empirical $iu6y of regression test selection techniques 

Todd L. Graves, Mary Jean Harrold, Jung-Min Kim, Adam Porter, Gregg Rothermel 

April 1998 Proceedings of the 20th international conference on Software engineering 

Full text available:. 



f|£^A;0S.MBi;S Additional Information: full ciiation . references, cities , index tes 
^•blisher Site 
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20 An empjri^ Q 
Todd L. Graves, Mary Jean Harrold, Jung-Min Kim, Adam Porter, Gregg Rothermel 
April 2001 ACM Transactions on Software Engineering and Methodology (TOSEM) 

Volume 10 Issue 2 

Full text available: H odft 169. 73 KB) Additional lnformation: Miration, abstract, references, citings, index 

' terms, review 

Regression testing is the process of validating modified software to detect whether new 
errors have been introduced into previously tested code and to provide confidence that 
modifications are correct. Since regression testing is an expensive process, researchers have 
proposed regression test selection techniques as a way to reduce some of this expense. 
These techniques attempt to reduce costs by selecting and running only a subset of the test 
cases in a program's existing test suite. Altho ... 

Keywords: empirical study, regression testing, selective retest 
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1 Automated-generating test case using UML statechart diagrams 
Supaporn Kansomkeat, Wanchai Rivepiboon 

September 2003 Proceedings of the 2003 annual research conference of the South 

African institute of computer scientists and information technologists 
on Enablement through technology 

Full text available: "j ^ srf K SO. 6 7 KB) Additional Information: Ml citation . abstract, reference;; , index terms 

Software testing plays a major role in software development because it accounts for a large 
part of the development cost. Moreover, manual testing technique always makes a problem. 
Consequently, this paper proposes the automatic testing technique to solve partially the 
testing process. This technique can automatically generate and select test cases from UML 
statechart diagrams. Firstly, we transform this diagram into intermediate diagram, called 
Testing Flow Graph (TFG), explicitly identify flow ... 

Keywords: UML specification, UML statechart diagrams, design, software testing, test 
generation 



Specjfjcat^^^^ 

Yanping Chen, Robert L. Probert, D. Paul Sims 

September 2002 Proceedings of the 2002 conference of the Centre for Advanced 
Studies on Collaborative research 

Full text available: ^^d£^8..89.K6j Additional Information: Mixtion, abstract, references, index Jenris 

Regression testing is essential to ensure software quality. The test team applies a regression 
test suite to ensure that new or modified features do not regress (make worse) existing 
features. Although existing research has addressed many problems and put forward 
solutions, most regression test techniques are code-based. Code-based regression test 
selection is good for unit testing, but it has a scalability problem. When the size of the 
subject under test grows, it becomes hard to ... 

Ur^LrBasedjntegra 

Jean Hartmann, Claudio Imoberdorf, Michael Meisinger 

August 2000 ACM SIGSOFT Software Engineering Notes , Proceedings of the 2000 ACM 
SIGSOFT international symposium on Software testing and analysis, volume 

25 Issue 5 

Additional Information: Mi.PMj.Qn, abstract, references, citings, index 



Full text available: TO.P.dfi.Sil,M.KS;. 



•jfros 



Increasing numbers of software developers are using the Unified Modeling Language (UML) 
and associated visual modeling tools as a basis for the design and implementation of their 
distributed, component-based applications. At the same time, it is necessary to test these 
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components, especially during unit and integration testing .At Siemens Corporate Research, 
we have addressed the issue of testing components by integrating test generation and test 
execution technology with commerci ... 

Keywords: COM/DCOM, CORBA, UML statecharts, distributed components, functional 
testing, test execution, test generation 



4 Ihe.AGED).S.toojs.for Q 
A. Hartman, K. Nagin 

July 2004 ACM SIGSOFT Software Engineering Notes , Proceedings of the 2004 ACM 
SIGSOFT international symposium on Software testing and analysis, volume 

29 Issue 4 

Full text available: ^.pdftgSO^S. KB). Additional Information: Mlcltation, abstract, references, .index. terms 

We describe the tools and interfaces created by the AGEDIS project, a European 
Commission sponsored project for the creation of a methodology and tools for automated 
model driven test generation and execution for distributed systems. The project includes an 
integrated environment for modeling, test generation, test execution, and other test related 
activities. The tools support a model based testing methodology that features a large degree 
of automation and also includes a feedback loop integrat ... 

Keywords: UML modeling, automated test generation, coverage analysis, defect analysis, 
test execution framework, validation 



5 SQfKvare^ Q 

models 

Alessandra Cavarra, Charles Crichton, Jim Davies 

March 2003 Proceedings of the 2003 ACM symposium on Applied computing 

Full text available: ^ »df(619. t 89 KB). Additional Information: fyjlcitatjon, sb^d:, reMsiLces 

This paper explains how object models written in the Unified Modeling Language (UML) can 
be translated into formal, behavioural descriptions and used as a basis for automatic test 
generation. The behavioural descriptions are written in a language of communicating state 
machines: the Intermediate Format (IF). The translation from UML to IF is based upon an 
earlier formal semantics, written in the Abstract State Machine (ASM) notation. Descriptions 
written in IF can be automatically explored; the ... 

Keywords: UML, formal methods, model based testing 



Middleware; Designing a test suite for empirically-based middleware performance 
prediction 

Yan Liu, Ian Gorton, Anna Liu, Ning Jiang, Shiping Chen 

February 2002 Proceedings of the Fortieth International Confernece on Tools Pacific: 
Objects for internet, mobile and embedded applications - Volume 10 

_ . , n , Additional Information: \m\\ citation , abstract. raferenc-ss . citings . index 



Full text available: WL sxlf£1 .03 MB) 

terms 

One of the major problems in building large-scale enterprise systems is anticipating the 
performance of the eventual solution before it has been built. This problem is especially 
germane to modern Internet-based e-business applications, where failure to provide high 
performance and scalability can lead to application and business failure. The fundamental 
software engineering problem is compounded by many factors, including application 
diversity, architectural trade-offs and options, COTS compone ... 

Keywords: COTS, component-based system, empirical results, middleware, performance 
modelling, prototype 
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Projected^ 

G. Friedman, A. Hartman, K. Nagin, T. Shiran 

July 2002 ACM SIGSOFT Software Engineering Notes , Proceedings of the 2002 ACM 
SIGSOFT international symposium on Software testing and analysis, Volume 

27 Issue 4 

Full text available: ^pdf(40o,24.KB) Additional Information: Ml .citation, abstract, references, indexternvs 

Our research deals with test generation for software based on finite state machine (FSM) 
models of the program specification. We describe a set of coverage criteria and testing 
constraints for use in the automatic generation of test suites. We also describe the 
algorithms used to generate test suites based on these coverage criteria, and the 
implementation of these algorithms as an extension of the Mur&phiv; model checker[4]. The 
coverage criteria are simple but powerful in that they generate te ... 

Keywords: automated test generation, finite state machine modeling, state machine 
projection., validation 



8 Technic^ 

Sebastian Eibaum, Srikanth Karre, Gregg Rothermel 

May 2003 Proceedings of the 25th international conference on Software engineering 

Full text available:,™ , 0?flDt S! 

^SfiTtA:j.9.?ylB A .*13*! Additional Information: Ml.cjMon, ai^tr&ct, references 
Publisher Site 

Web applications have become critical components of the global information infrastructure, 
and it is important that they be validated to ensure their reliability. Therefore, many 
techniques and tools for validating web applications have been created. Only a few of these 
techniques, however, have addressed problems of testing the functionality of web 
applications, and those that do have not fully considered the unique attributes of web 
applications. In this paper we explore the notion that user s ... 

9 Stei|c.and. dynamic 

Atanas Rountev, Scott Kagan, Michael Gibas 

July 2004 ACM SIGSOFT Software Engineering Notes , Proceedings of the 2004 ACM 
SIGSOFT international symposium on Software testing and analysis, volume 

29 Issue 4 

Full text available: ^eg£2?.8.MJ<Bj Additional Information: Ml .cjtation, abstract, references, indextenris. 

This work presents a parameterized framework for static and dynamic analysis of call chains 
in Java components. Such analyses have a wide range of uses in tools for software 
understanding and testing. We also describe a test coverage tool built with these analyses 
and the use of the tool on a real-world test suite. Our experiments evaluate the exact 
precision of several instances of the framework and provide a novel approach for estimating 
the limits of class analysis technology for computing pr ... 

Keywords: call chains, call graph, dynamic analysis, static analysis 



10 Technical papers; testing Si: A framework for component deployment testing 
Antonia Bertolino, Andrea Polini 

May 2003 Proceedings of the 25th international conference on Software engineering 

Full text available:,™ ^^[Sl 

^SSllL^M.*;..^ Additional Information: MLdMlo.0., abstract, refeiences 

Component-based development is the emerging paradigm in software production, though 
several challenges still slow down its full taking up. In particular, the "component trust 
problem" refers to how adequate guarantees and documentation about a component s 
behaviour can be transferred from the component developer to its potential users. The 
capability to test a component when deployed within the target application environment can 
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help establish the compliance of a candidate component to the cust ... 

1 1 Analysis. and Jesting, of 
Filippo Ricca, Paolo Tonella 

July 2001 Proceedings of the 23rd international conference on Software engineering 

Full text available: 'gj prfK -=57.58 KB; Additional Information: foil citation , abstract, references , citings , index 

®.PuysherSjte teEIOS. 

The economic relevance of Web applications increases the importance of controlling and 
improving their quality. Moreover, the new available technologies for their development 
allow the insertion of sophisticated functions, but often leave the developers responsible for 
their organization and evolution. As a consequence, a high demand is emerging for 
methodologies and tools for quality assurance of Web based systems. 

In this paper, a UML model of Web applications is proposed for their ... 

Keywords: UML modeling, code analysis, reverse engineering, testing, web applications 



1 2 Web technojogi.es Q 
applications 

Carlo Bellettini, Alessandro Marchetto, Andrea Trentini 

March 2004 Proceedings of the 2004 ACM symposium on Applied computing 

Full text available: ^.p.d£631. J.3 KB). Additional Information: Mlvi&iteEi, abstract, fgM§ILces 

Web applications have become complex and crucial for many firms, especially when 
combined with areas such as CRM (Customer Relationship Management) and BPR (Business 
Process Reengineering). Since then the scientific community has focused attention to Web 
application design, development, analysis, testing, by studying and proposing methodologies 
and tools. This paper describes an automatic tool for the construction of UML models from 
existing Web applications. This tool, named WebUml, generates c ... 

13 ProgrammingJa^ 

objection 
Issa Traore 

March 2003 Proceedings of the 2003 ACM symposium on Applied computing 

Full text available: ^fidjQ J>PJvlBl Additional Information: Ml.£iMon, ?;bslrad, inferences, ciijngs 

Though time-to-market has become the primary criterion that drives most current software 
development projects, quality still remains the key concern of critical software development 
projects, for which the cost of a single bug may involve serious loss or damages. Meeting 
the higher quality level required for such kinds of systems may be achieved only by using 
sound and rigorous test practices. We present in this paper an integrated platform that uses 
a formalized version of UML statechart as the ... 

Keywords: PVS, UML, formal methods, object technology, object-oriented testing, 
requirements specification 

14 ELI workshopsi World 
Ulf Wiger, Gosta Ask, Kent Boortz 

December 2002 ACM SIGPLAN Notices, volume 37 issue 12 

Full text available: c-^ff 195.51 KB) Additional Information: foil Citation, abstract, referertcfts . index terms 

It is now ten years ago since the decision was made to apply the functional programming 
language Erlang to real production projects at Ericsson. In late 1995, development on the 
Open Telecom Platform (OTP) started, and in mid 1996 the AXD 301 project became the 
first user of OTP. The AXD 301 Multi-service Switch was released in October 1998, and later 
became "the heart of ENGINE", Ericsson's leading Voice over Packet solution. In those early 
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15 World-class product certification using Erlang 
Ulf Wiger, Gosta Ask, Kent Boortz 

October 2002 Proceedings of the 2002 ACM SIGPLAN workshop on Erlang 

Full text available: f§ odK14K.2S KB) Addltional Information: M.cltation, abstract, references, citings, Index 

It is now ten years ago since the decision was made to apply the functional programming 
language Erlang to real production projects at Ericsson. In late 1995, development on the 
Open Telecom Platform (OTP) started, and in mid 1996 the AXD 301 project became the 
first user of OTP. The AXD 301 Multi-service Switch was released in October 1998, and later 
became "the heart of ENGINE", Ericsson's leading Voice over Packet solution. In those early 
days of Erlang programming, high-level tools for develo ... 

Keywords: erlang, testing 



16 Evolutionary design of complex software (EDCS) demonstration days 1999 Q 
Wayne Stidolph 

January 2000 ACM SIGSOFT Software Engineering Notes, volume 25 issue l 
Full text available: ^fidfCLSO.jylBi Additional Information: .Mi .citation, abstract, index terms 

This report summarizes the Product/Technology demonstrations given at Defense Advanced 
Research Projects Agency (DARPA) Evolutionary Design of Complex Software (EDCS) 
Program Demonstration Days, held 28-29 June 1999 at the Sheraton National Hotel, 
Arlington, VA. 

March 2003 ACM SIGSOFT Software Engineering Notes, volume 28 issue 2 
Full text available: 'P ) sdfi 346.94 KB) Additional Information: foil citation 



18 Automated testing of POSiX standards ' Q 

J. F. Leathrum, K. A. Liburdy 
March 1994 StandardView, volume 2 issue l 

Full text available: ^.pd£468.67. KB). Additional Information: fujj. citation, references, indexlenm 



19 Object-oriented technology: Maintaining software through intentional source-code * 
views 

Kim Mens, Tom Mens, Michel Wermelinger 

July 2002 Proceedings of the 14th international conference on Software engineering 
and knowledge engineering 

Full text available: ^|^lf(2feL9&J<Bj Additional Information: ML£l3tt»tt, is&str&ct, refor^rxoes 

Maintaining the source code of large software systems is hard. One underlying cause is that 
existing modularisation mechanisms are inadequate to handle crosscutting concerns. We 
propose intentional source-code views as an intuitive and lightweight means of modelling 
such concerns. They increase our ability to understand, modularise and browse the source 
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